A Character String-Based Stemming for Morphologically Derivative Languages

Morphologically derivative languages form words by fusing stems and suffixes, are important to be extracted in order make cross lingual alignment knowledge transfer. As there phonetic harmony disharmony when linguistic particles combined, both morphological changes need analyzed. This paper proposes a multilingual stemming method that learns morpho-phonetic automatically based on character embedding sequential modeling. Firstly, the feature at sentence level is used as input, BiLSTM model obtain forward reverse context sequence, attention mechanism added this for weight learning, global information capture stem affix boundaries; finally CRF learn more from sequence features describe effectively. In verify effectiveness of above model, compared with traditional two different data sets three languages: Uyghur, Kazakh Kirghiz. The experimental results show has best effect sentence-level datasets, which leads effective stemming. addition, proposed outperforms other models, fully consider characteristics, certain advantages less human intervention.